Robust boosting for regression problems

Gradient boosting algorithms construct a regression predictor using linear combination of “base learners”. Boosting also offers an approach to obtaining robust non-parametric estimators that are scalable applications with many explanatory variables. The algorithm is based on two-stage approach, similar what done for regression: it first minimizes residual scale estimator, and then improves by optimizing bounded loss function. Unlike previous proposals this does not require computing ad hoc estimator in each iteration. Since the functions involved typically non-convex, reliable initialization step required, such as L1 tree, which fast compute. A variable importance measure can be calculated via permutation procedure. Thorough simulation studies several data analyses show that, when no atypical observations present, works well standard gradient squared loss. Furthermore, contain outliers, outperforms alternatives terms prediction error selection accuracy.